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HUMAN RELIABILITY ANALYSIS (HRA) OVERVIEW 



• HRA is a method used to describe, qualitatively and quantitatively, the 
occurrence of human failures in the operation of complex systems that 
affect availability and reliability. 

• Modeling human actions with their corresponding failure in a PRA 
(Probabilistic Risk Assessment) provides a more complete picture of the 
risk and risk contributions. 

• A high quality HRA can provide valuable information on potential areas for 
improvement, including training, procedural, equipment design and need 
for automation. 

- For Shuttle, the HRA was useful to show the importance of maintaining crew training at 
it's current level in a time when budget reductions were threatening training levels. 

- For Shuttle, the HRA showed areas where automation would be beneficial but 
considering the Shuttle program retirement, it was not worth pursuing (e.g. H20 loop 
freeze protection given overcool event). 


2 


SPACE SHUTTLE PROGRAM 

Space Shuttle Safety and Mission Assurance Office 

NASA Johnson Space Center, Houston, Texas 

HUMAN RELIABILITY ANALYSIS (HRA) OVERVIEW (2) 



• Modeling human error has always been a challenge 

— Performance data is not always readily available 

— When data is available , it is sensitive and must be handled with care 

• For spaceflight, the challenge is amplified 

— small number of participants 

— limited amount of performance data available 

— lack of definition of the unique factors influencing human performance in 
space 

• Performance Shaping Factors (PSF), in HRA terminology, are used in HRA techniques 
to modify basic human error probabilities in order to capture the context of an 
analyzed task. 

• Many of the human error modeling techniques were developed within the context 
of nuclear power plants and therefore the methodologies do not address 
spaceflight factors, such as the effects of microgravity and long duration missions. 
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PHILOSOPHY FOR MODELING HUMAN ACTIONS 



• Model human actions that are required for normal operation of a system 

• In general, recoveries and work-a-rounds are only modeled if the scenario 
becomes dominate in the risk profile 

• Use screening Human Error Probabilities (HEPs) for recoveries and work-a- 
rounds in order to reduce the number events required for detailed 
analysis down to only those that are significant 

Screening analysis should be quick and inherently conservative so that lack of 
detailed modeling does not lead to underestimation of the risk 

• There are many methodologies for performing HRA, NASA JSC has chosen 
to use an internally developed screening method and the Cognitive 
Reliability Error Analysis Model (CREAM) 

- Key Factors for Selecting an HRA Approach 

• Ability to model errors of commission and continuous feedback events as well as 
errors of omission 

• Reproducibility of results 

• Reasonable results 

• Ability to perform the analysis in-house in a reasonable timeframe 
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EXPECTED EFFORT FOR PERFORMANCE OF HRA 



• Screening analysis should take < 1 hour per event 

- Can be performed by either HRA lead or individual analyst. If 
performed by analyst, it should be reviewed by HRA lead to ensure 
consistency 

• Detailed analysis ~3 to 5 days per event 

- Performed by a single HRA lead 

- Requires research 

- Interviews with Astronauts, Mission Operations, Trainers, interaction 
with the PRA system's analyst 

For Shuttle PRA, ~70% of HRA events are modeled with Screening Analysis 
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EXAMPLE HRA SCREENING TABLE 



(Means and Lognormal Distributions) 


Available Time 

Ideal HEP 

Nominal HEP 

1 Adverse 
Condition 

2 Adverse 
Conditions 

3 Adverse 
Conditions 

4 Adverse 
Conditions 

T < 1 minute 

0.16 
EF 5 

0.48 
EF 5 

1 

1 

1 

1 

T > 1 minute 
T < 10 minutes 

0.048 
EF 5 

0.16 
EF 5 

0.48 
EF 5 

1 

1 

1 

T > 10 minutes 
T < 30 minutes 

0.016 
EF 5 

0.048 
EF 5 

0.16 
EF 5 

0.48 
EF 5 

1 

1 

T > 30 minutes 

0.0048 
EF 5 

0.016 
EF 5 

0.048 
EF 5 

0.16 
EF 5 

0.48 
EF 5 

1 


Adverse conditions are degraded or extreme environments, unfamiliar tasks, 
high stress scenarios, and complex tasks 


Screening values are used to determine whether a detailed calculation is worth 
pursuing 6 
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SHUTTLE PRA ITERATION 3.2 CONTRIBUTIONS BY 
ELEMENT OR MAJOR AREA* 




MMOD 


Orbiter Hardware/ 
Software 


SSME 


Human Error 


Ascent Debris 


SRB 


RSRM 


External Tank 


1.E-05 


1:6000 


1.E-04 




1:1500 


1:300 


1:350 


1:610 
1:770 


:03O 



1.E-03 


Scenarios 
involving 
human error 
provide a 
significant risk 
contribution 


1.E-02 


1.E-01 


* Some overlap in risk exists. For example, a cut set containing both a mechanical failure and a human error that 
result in failure to lower the landing gear is counted in both the Orbiter hardware contributor and the human error 
contributor. 7 
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SHUTTLE PRATOP CONTRIBUTORS INVOLVING HUMAN ERROR 


Rank 

%age of 
Total 

Cumulativ 
e Total % 

Probability 

Description 

1 

7.3 

7.3 

8.2E-04 
(1 in 1200) 

Crew error during entry 

2 

1.2 

8.5 

1.3E-04 
(1 in 7700) 

Collision of the Orbiter with the International Space Station (ISS) during 
rendezvous and docking 

3 

0.9 

9.4 

1.0E-04 
(1 in 9500) 

Orbiter inspections (Flight Day 2 and late) produce false positive indications of 
damage, and failure of crew rescue 

4 

0.7 

10.1 

7.5E-05 
(1 in 13,000) 

MPS component failures cause a catastrophic overpressure condition in the aft 
compartment during entry 

5 

0.3 

10.4 

3.3E-05 
(1 in 30,000) 

Fuel cell leak and a subsequent failure of the crew to respond appropriately 
causes a catastrophic failure 

6 

0.3 

10.7 

3.1E-05 
(1 in 32,000) 

Orbit inspections (Flight Day 2 and late) result in damage to the TPS 

7 

0.2 

10.9 

1.9E-05 
(1 in 51,000) 

Cabin depressurization due to leaks beyond the make-up capability of the 
Pressure Control System (e.g., penetration leaks) or pressure control system fails 

8 

0.1 

11.1 

1.2E-05 
(1 in 81,000) 

APU heater fails on and human error failure results in catastrophic failure on orbit 

9 

0.1 

11.2 

1.2E-05 
(1 in 83,000) 

Failure of Deorbit burn due to improper targeting of OMS burn (human error) 

10 

0.1 

11.3 

9.9E-06 
(1 in 100,000) 

Cabin Fan System failure combined with a human error during landing brought 
about by high heat or humidity 

11 

0.1 

11.4 

6.6E-06 
(1 in 150,000) 

Landing Deceleration System (LDS) tire ruptures 

12 

0.1 

11.5 

6.6E-06 
(1 in 150,000) 

Flash Evaporator System freeze up and failure to recover leads to LOCV during 
entry 


Scenarios with some applicability to commercial crew. The particular scenario may not be 100% 
applicable but similar scenarios would exist (for example may not have an APU heater but may 
have a heater scenario that could lead to LOC if not mitigated by crew) 
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INSIGHTS FROM SHUTTLE PRA APPLICABLE TO 
COMMERCIAL SPACE FLIGHT 



• Helpful to have a single HRA lead which models Human error events to 
maintain consistency 

- Less important on screening events but necessary for events modeled in detail 

• Special attention should be paid to critical crew/human actions required for 
a nominal flight 

- Contributed to ~10% of the overall Shuttle risk with the other 2% human error 
contribution relating to responses to failures 

• If it is unknown whether or not an action is to be automated, evaluate it as 
manual 

- Difficult to identify later on, and will lead to underestimates of the risk if action 
is actually manual 

• Use of CREAM methodology only slightly modified to capture dependencies 
and uncertainty may be reasonable for short duration missions such Shuttle 
mission of 11-16 days. However, the space environment may be a more 
important factor for longer duration missions of 6 months or more 
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PLANS FOR EXPANDING CURRENT HRA METHODOLOGY 



• First step is working on identifying and quantifying a new set of 
Performance Shaping Factors which are relevant to spaceflight 

- Recommendation from NASA 2006 HRA TIM was to address Performance 
Shaping Factors (PSFs) specific to spaceflight 

- Graduate Student from University of Colorado at Boulder primarily funded by 
NASA Graduate Student Researchers Program (GSRP) is work ing with JSC 

• Selected PSFs quantified by the end of 2011, with expected follow on work to be 
completed by NASA 

• Second step is working on identifying and quantifying a new set of basic 
human error probabilities with a potential for a new cognitive model 

- Work with University of Maryland to address limitations of current HRA model 
with respect to basic human error probabilities and cognitive model which are 
more focused on ground based activities 

- Less effort has been spent on this activity to date 
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EXAMPLE OF PSFS AND WEIGHT FACTORS 




(COGNITIVE RELIABILITY AND ERROR ANALYSIS METHOD - CREAM) 


COMMON PERFORMANCE CONDITIONS WEIGHT FACTORS 


CPC Name Level Cognitive Function 

Observation Interpretation Planning Execution 


From CREAM: 


Adequacy of organization 

Very efficient 

1.0 

1.0 

0.8 

0.8 

Efficient 

1.0 

1.0 

1.0 

1.0 

Inefficient 

1.0 

1.0 

1.2 

1.2 

Deficient 

1.0 

1.0 

2.0 

2.0 

Working conditions 

Advantageous 

0.8 

0.8 

1.0 

0.8 

Compatible 

1.0 

1.0 

1.0 

1.0 

Incompatible 

2.0 

2.0 

1.0 

2.0 

Adequacy of MMI and 
operational support 

Supportive 

0.5 

1.0 

1.0 

0.5 

Adequate 

1.0 

1.0 

1.0 

1.0 

Tolerable 

1.0 

1.0 

1.0 

1.0 

Inappropriate 

5.0 

1.0 

1.0 

5.0 

Availability of 
procedures/plans 

Appropriate 

0.8 

1.0 

0.5 

0.8 

Acceptable 

1.0 

1.0 

1.0 

1.0 

Inappropriate 

2.0 

1.0 

0.5 

2.0 

Number of simultaneous 
goals 

Fewer than capacity 

1.0 

1.0 

1.0 

1.0 

Matching current capacity 

1.0 

1.0 

1.0 

1.0 

More than capacity 

2.0 

2.0 

5.0 

2.0 

Available time 

Adequate 

0.5 

0.5 

0.5 

0.5 

Temporarily inadequate 

1.0 

1.0 

1.0 

1.0 

Continuously inadequate 

5.0 

5.0 

5.0 

5.0 

Time of day (circadian 
rhythm) 

Day-time (adjusted) 

1.0 

1.0 

1.0 

1.0 

Night-time (unadjusted) 

1.2 

1.2 

1.2 

1.2 

Adequacy of training and 
experience 

Adequate, high experience 

0.8 

0.5 

0.5 

0.8 

Adequate, low experience 

1.0 

1.0 

1.0 

1.0 

Inadequate 

2.0 

5.0 

5.0 

2.0 

Crew collaboration quality 

Very efficient 

0.5 

0.5 

0.5 

0.5 

Efficient 

1.0 

1.0 

1.0 

1.0 

Inefficient 

1.0 

1.0 

1.0 

1.0 

Deficient 

2.0 

2.0 

2.0 

5.0 





Reference: Hollnagel, E. Cognitive Reliability and Error Analysis Method (CREAM). Elsevier Science. 1998. 


PSFs that 
may need to 
be modified 
for space 
applications 


First Step: Expand and Modify These PSFs from CREAM 
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CREAM QUANTITATIVE PERFORMANCE PREDICTION 



Is task mainly observation, interpretation, planning, or execution ? 
Identify the likely cognitive function failure 

Determine the 
failure probability by 
using nominal failure 
value given for each 
function failure. 

Adjust values based 


Reference: Hollnagel, E. Cognitive Reliability and Error Analysis Method (CREAM). Elsevier Science. 1998 

Second Step: Identify and Quantify New basic human error 

probabilities with the potential for adopting new 
cognitive model 12 


on performance 
shaping factors 
(PSFs) 


Cognitive 

Function 

Generic Failure Type 

Lower Bound 
(5 percentile) 

Basic Value 

Upper Bound 
(95 percentile) 

Observation 

Ol. Wrong object observed 

3.0E-4 

1.0E-3 

3.0E-3 

02. Wrong identification 

1.0E-3 

3.0E-3 

9.0E-3 

03. Observation not made 

1.0E-3 

3.0E-3 

9.0E-3 

Interpretation 

11. Faulty' diagnosis 

9.0E-2 

2.0E-1 

6.0E-1 

12. Decision error 

1.0E-3 

1.0E-2 

l.OE-1 

13. Delayed interpretation 

1.0E-3 

1.0E-2 

l.OE-1 

Planning 

PI. Priority error 

1.0E-3 

1.0E-2 

l.OE-1 

P2. Inadequate plan 

1.0E-3 

1.0E-2 

l.OE-1 

Execution 

El. Action of wrong type 

1.0E-3 

3.0E-3 

9.0E-3 

E2. Action at -wrong time 

1.0E-3 

3.0E-3 

9.0E-3 

E3. Action on wrong object 

5.0E-5 

5.0E-4 

5.0E-3 

E4. Action out of sequence 

1.0E-3 

3.0E-3 

9.0E-3 

E5. Missed action 

2.5E-2 

3.0E-2 

4.0E-2 
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BACKUP 
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COMPARISON OF CREAM TO SIMULATOR DATA 


“Land Too Hard” 


“Failure to Lower 
Landing Gear” 


“Brake at Wrong 
Time” 


CREAM 

SIM 

CREAM 

SIM 

CREAM 

SIM 



1.E-06 


1 .E-05 


1 .E-04 


1.E-03 


For available data, CREAM compared well with Shuttle simulator 
data 


1 .E-02 


-Added to credibility of the analysis 
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CREAM QUANTITATIVE PERFORMANCE PREDICTION 


• Is task mainly observation, interpretation, planning, or execution ? 

• Identify the likely cognitive function failure. 

• Determine the failure probability by using 
nominal failure value 


given for each 
function failure. 


Cognitive 

Function 

Generic Failure Type 

Lower Bound 
(5 percentile) 

Basic Value 

Upper Bound 
(95 percentile) 

Observation 

Ol. Wrong object observed 

3.0E-4 

1.0E-3 

3.0E-3 

02. Wrong identification 

1.0E-3 

3.0E-3 

9.0E-3 

03. Observation not made 

1.0E-3 

3.0E-3 

9.0E-3 

Interpretation 

11. Faulty diagnosis 

9.0E-2 

2.0E-1 

6.0E-1 

12. Decision error 

1.0E-3 

1.0E-2 

l.OE-1 

13. Delayed interpretation 

1.0E-3 

1.0E-2 

l.OE-1 

Planning 

PI. Priority error 

1.0E-3 

1.0E-2 

l.OE-1 

P2. Inadequate plan 

1.0E-3 

1.0E-2 

l.OE-1 

Execution 

El. Action of wrong type 

1.0E-3 

3.0E-3 

9.0E-3 

E2. Action at wrong time 

1.0E-3 

3.0E-3 

9.0E-3 

E3. Action on wrong object 

5.0E-5 

5.0E-4 

5.0E-3 

E4. Action out of sequence 

1.0E-3 

3.0E-3 

9.0E-3 

E5. Missed action 

2.5E-2 

3.0E-2 

4.0E-2 


Adjust values based on performance shaping factors (PSFs) 

Reference: Hollnagel, E. Cognitive Reliability and Error Analysis Method (CREAM). Elsevier Science. 1998. 
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CREAM QUANTITATIVE EXAMPLE 

Basic Event: Collision of Orbiter with International Space Station (ISS) during 
rendezvous and docking 

Five cognitive activities were identified for this action 

- EXECUTE - The crew executes rendezvous and docking actions. 

- OBSERVE - The crew observes an erroneous action has been taken. 

- OBSERVE - The MCC observes an erroneous action has been taken. 

- EXECUTE - The MCC fails to warn crew of the need to recover. 

- EXECUTE - The crew recovers from erroneous action 

Each cognitive activity is matched to its dominant cognitive function and likely 
failure. 


Cognitive Activity 

Cognitive Function 

Predominant Failure 

Nominal Failure 
Probability (Median) 

Execute 

Execution 

Action of a Wrong Type (Shuttle crew) 

3.0E-03 

Observe 

Observation 

Observation not made (Shuttle crew) 

3.0E-03 

Observe 

Observation 

Observation not made (MCC crew) 

3.0E-03 

Execute 

Execution 

Missed Action (MCC crew) 

3.0E-02 

Execute 

Execution 

Action at the Wrong Time (Shuttle crew) 

3.0E-03 


Reference: Hamlin, T. Space Shuttle Program Human Reliability Analysis (HRA) Data Report. Nov. 2008. 
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CREAM QUANTITATIVE EXAMPLE (2) 



• Once each predominant cognitive failure is identified, the Performance Shaping 
Factors (PSFs) are evaluated. 

• Evaluated for the Shuttle crew and MCC crew separately. 

• The weighting factors corresponding to the PSF evaluations are found: 


PSF Name 

Level 

Exec 

Wrong 

Type 

Obs Not 
Made 

Obs Not 
Made 
(MCC) 

Exec 

Missed 

(MCC) 

Exec 

Wrong 

Time 

A 

Adequacy of Organization 

Very Efficient 

0.8 

1 

1 

0.8 

0.8 

B 

Working Conditions 

Compatible/Advantageous 

1 

1 

0.8 

0.8 

1 

C 

Adequacy of MMI 

Supportive/ Adequate 

0.5 

1 

0.5 

0.5 

1 

D 

Procedures/Plans 

Appropriate 

0.8 

0.8 

0.8 

0.8 

0.8 

E 

Number of Goals 

Fewer Than Capacity 

1 

1 

1 

1 

1 

F 

Available Time 

Adequate/ 

Continuously Inadequate 

0.5 

5 

5 

5 

5 

G 

Time of Day 

Daytime 

1 

1 

1 

1 

1 

H 

Training & Preparation 

Adequate, High Experience/Adequate, 
Low Experience 

0.8 

1 

1 

1 

1 

1 

Crew Collaboration 

Very Efficient 

0.5 

0.5 

0.5 

0.5 

0.5 

Total Influence of PSFs = A*B*C*D*E*F*G*H*I 
(overall weighting factor) 

0.064 

2 

0.8 

0.64 

1.6 


Reference: Hamlin, T. Space Shuttle Program Human Reliability Analysis (HRA) Data Report. Nov. 2008. 


18 


SPACE SHUTTLE PROGRAM 

Space Shuttle Safety and Mission Assurance Office 

NASA Johnson Space Center, Houston, Texas 

CREAM QUANTITATIVE EXAMPLE (3) 



• The weighting factors are combined with the nominal failure probabilities to 
obtain adjusted values: 


Activity 

Predominant 

Failure 

Nominal Failure 
Probability 

Overall Weighting 
Factor 

Adjusted Failure 
Probability 

Adjusted for 
Dependency 

Crew Error During 
Docking 

Action of a Wrong 
Type (Shuttle crew) 

3.0E-03 

0.064 

1 .9E-04 Median 
(2.4E-04 Mean) 

N/A 

Crew Fails to Observe 
need for recovery 

Observation not 
made (Shuttle crew) 

3.0E-03 

2 

6.0E-03 Median 

0.55 Mean 
(High Depend) 

MCC fails to Observe 
need for recovery 

Observation not 
made (MCC crew) 

3.0E-03 

0.8 

2.4E-03 Median 

0.19 Mean 
(Medium Depend) 

MCC fails to Warn 
Crew 

Missed Action (MCC 
crew) 

3.0E-02 

0.64 

1 .9E-02 Median 

0.21 Mean 
(Medium Depend) 

Crew Fails to Execute 
Recovery 

Action at the Wrong 
Time (Shuttle crew) 

3.0E-03 

1.6 

4.8E-03 Median 

0.55 Mean 
(High Depend) 


• Dependencies between the Shuttle Crew and MCC crew are then taken into 
account in combining the activity probabilities 

• Leads to overall probability for the event “Collision of Orbiter with 
International Space Station (ISS) during rendezvous and docking” 

- Max probability => 2.4E-04 * 0.55 = 1.3E-04 


Reference: Hamlin, T. Space Shuttle Program Human Reliability Analysis (HRA) Data Report. Nov. 2008. 
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